-
Notifications
You must be signed in to change notification settings - Fork 182
Break PostResponse requestcontrol
plugin into 3 separate plugins to add streamed request functionality
#1661
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Break PostResponse requestcontrol
plugin into 3 separate plugins to add streamed request functionality
#1661
Conversation
✅ Deploy Preview for gateway-api-inference-extension ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Hi @BenjaminBraunDev. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
/ok-to-test |
Some smaller comments, overall looks good. Thanks! /approve |
} | ||
|
||
// HandleResponseBodyStreaming is called every time a chunk of the response body is received. | ||
func (d *Director) HandleResponseBodyStreaming(ctx context.Context, reqCtx *handlers.RequestContext, logger logr.Logger) (*handlers.RequestContext, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please remove logger from the args, we don't use logr as argument to functions, we use contextual logging.
for example
log.FromContext(ctx).V(logutil.TRACE).Info(....)
see for example here:
loggerTrace := log.FromContext(ctx).V(logutil.TRACE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I asked him to remove that, in the streaming case the high volume of instantiations of the logger will cause allocation/gc pressure. Luke mentioned this as an optimization during flow control benchmarking, and i think the same applies here b/c there will be multiple streaming calls per request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does log.FromContext(ctx).V(logutil.TRACE).Info(....)
actually make another logger? I would think it takes the logger "from the context" given ctx has a logger. Does it actually allocate more memory? I would think that pretty inefficient.
From the FromContext() in go-logr package:
// FromContext returns a Logger from ctx or an error if no Logger is found.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nirrozenbaum @kfswain What is the desired state for this? I can revert it back or keep the changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it makes another logger.
that wouldn't make sense.
the desired state (at least in my opinion) is to remove logger from the args and use contextual logging if/when necessary.
we don't pass around logger in any part of the code. we pass ctx (which includes inside it the logger).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, keeping logger in parameters. I don't have a strong opinion on this beyond we probably shouldn't create a logger for every chunk streamed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a confusion here.
@kfswain the call Logger.WithValues
creates a new instance of the logger (with some key value pairs attached to it). we create a logger only ONCE somewhere up in the handling chain and store it inside the context using log.IntoContext(ctx, logger)
. for example here:
ctx = log.IntoContext(ctx, logger) |
then, in all other functions, we call log.FromContext(ctx)
. This call does NOT create a new logger. it retrieves the stored logger from the context. more information here:
https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/log#pkg-functions
The best practice in contextual logging is to create logger once and store it into the context, and retrieve it from context in the various places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yeah sorry, mixed messaging here, the initial implementation had the WithValues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm slightly confused here, the FromContext
in log.go always calls log.WithValues(keysAndValues...)
, it's the only way the function can return. Should I keep this logging regardless?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was checking now again. I see your point.
I’m still not sure about that, but anyway this looks to me like a system wide issue cause we’re using contextual logging in all places.
I suggest to move forward with current PR using Log.FromContext, open a dedicated issue to check about this issue, and if we decide to change, we need to update everywhere, not only in this function.
let me know if it makes sense.
@BenjaminBraunDev made another iteration on the PR. |
/approve Thanks Ben! Will let Nir give final stamp |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: BenjaminBraunDev, kfswain The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Co-authored-by: Nir Rozenbaum <[email protected]>
Co-authored-by: Nir Rozenbaum <[email protected]>
Co-authored-by: Nir Rozenbaum <[email protected]>
2a25c1a
to
f73ef27
Compare
/kind bug
/kind feature
What this PR does / why we need it:
PostResponse runs at the first chunk received instead of at the end of the request, however some PostResponse plugins would like to run only when a request is fully complete. This PR allows that flexibility by breaking out PostResponse into 3 separate requestcontrol plugin hooks for when the response is received, each chunk streamed, and fully complete respectively. The requestcontrol plugins are now the following:
Which issue(s) this PR fixes:
Fixes #1483
Does this PR introduce a user-facing change?: